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ABSTRACT 



Objective scores and judges ■ ratings, both basetf on 
complex personal assessment battery, were correlated with a" 
^upil-gain criterion of teaching effectiveness and with a number o.f . 
observationally deifivcd measures of classroom teaching behavior. 
Twenty-seven primary teachers were . studijed who had demonstrated 
consistent patterns of pupil gain avei the previous three years, 
varying substantially in the degree of gain achieved. The 
relationships of the assessment-based measures to the teaching 
effectiveness measures are described. In addition, the objective 
assessment scbres^and the judigmental ratings are compared in their 
predictive power. (Author) ' • 
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personal Characteristics Associate^ with Effective Teaching 

Robert T. Peck and Donald J, Veldman' . 
The .University of Texas at Austin 

» * . 

The objectives of this^phase of the research program were to identify 
individual* characteristics of attitude, conceptual style, and personality 
that may be significantly, related to teathing effectiveness. ,The measures 
used here may be vi3wed as "distal'' variables with regard to 'a criterion of 
effective impact on pupil, achievement, in contrast to the more "proximal" 
variables obtained^ by direct classroom observation of the teachers Concerned 
(seef papers by Brophy and Evertsdn). These distal variables were derived 



from paper/pencil ia&truments completed by the teachers themselves at the 
end of the school year following .the four years during, which their effectivje 



ne's;^* was ,asse3.sed in terms of pupil gains on standard achievement' measures* 

ft. 

Of particular J.nti^rest is the ii^clusion in this battery df both highly , 

structured (questionnaire) aqd relatively unstructured (f ree-respons^ 

instruments. Predictive estimates were^^^derived from these dati by objective 

scoring of^^each instrument separately and also by , ratings based on the total 
• » 

assessment file of each teacher* Both ^the^objeccive scores and the ratings > 

were then correlated with the "proximal" observation^based variables, as 

veil as with the pupil-*gain effectiveness criterion. « 

* « 

Sample and Measures 
♦ • 

Twenty'-seven teachers who had been selected as' "consistent" in thel^ 

. . • ^ ' ^ ' ^ 

relative effectiveness in eliciting pupil achievement gains (see paper by 

Veldman and Brophy), complet;^d a bat^tery o£ paper/pencil Instruments which 

included the following: 



t 




• • 


> 


« • • • • ' ' • • • 


* 




Questionnaires * * ' 




- 


Adjective Self -Description (Veldman and Parkeri^l970) • Yields seven . ^ 






trait-factor scores* ^ . • ' ' 


- 


/ . ■ . 


'.Self-Report Inventory (Sown and Richek, • 1967) • Yields eight attitude 
measures. - ^ 




« 

/ 






/iews of Life (Peck-, Diaz-Guerrerb and Bawrence, 1973). Yields 20 scale 




I * * 


scores* . * 

Free- Response Instruments \^ . ' . • e * . . • * . 
Biographical Information. Covers family- education, health, interests* 






work history, knd self-assessment of attributes' and goals)^ 






Concerns Statement (Puller, 1969). A listing of current personal teaching ' . . 






concerns. • • * 




• 


.Directed Imagination (Veldman and* Menaker,' 1969). Write three fictional ^ 
stories about teaching in 15 minutes. ' * ' 

One-Word Sentence Completion (Veldman, Menaker,'ahd Peel:, 1^969). Sibctv- 
tvo item form' requiring single-word responses. ^ ' * . . 
These teachers were selected on the basis of pupil-gain data' from years 1-3. 
Each teacher had a pupil-gain score for years 1-4, and an average score for 

the four years. The personality data were collected in the fifth year. 

< 


X 

1 




Procedure 


t 

i 
\ 




Each instrument was scored on a number of ^dimensions, according to 




published procedures. The individual file for each teacher, containing all 






of the original (unscored) instruments,'^ was^ thereafter independently evaluated 


i 

I ' 




by three experienced judges. Estimates of probable teaching behavior, based 


i 




on these evaluations, were recorded on the following seven-point rating scales: 


i 

i 

* 

8 




1. Subject-mart ar competence (Inter judge reliability .68) 


i 

\ 

1 ■ 

** 


ERIC 


2. Communication skills (.35) 

3. •Stimulating .imagination (.59) V *^ 



4* ' Responsible Independence (.58) , 
5» Poise and self-confidence (.83) 
, '6*' Attitude- toward students (#53) . 

7. Teaching style (teacher vs» pupil'^center^d) (.41) 
8» Dlscl^illMry control (.58) , 
9» Attitude toward supervision .(.59) / 



/ 



10* Reaction to own mistakes 

1^1. Alertness to classroom events (.59) / 

12. Professional cpnanitment (.39) 

./ 

13. Summary rating of competence (.60) / 



, 14. Ranking dn competence (.71) 



/ 



- In addition to the ratlngSy each jud^e /rank-bordered all 27 folder^ according 
to general levels of teaching ef f ectlvenes^ to be expected from the tochers. * ^ 
Inter^judge reliability estimates were then comi>jfc^t€^ among- the evaluations of 
the three judges^ and composite evaluations were*also computed. 

' The .final stage of the procedure was the calculation of correlations 
between the variables obtained by objective and clinical scoring of these 
distal- data and the assessments made from proximal classroom observatiop; with 
ratings tased on. short Interviews with the teachers;. and with the teaching 
effectiveness criteria derived from analysis of pupil achievement data. 
Findings 

Table 1 shows that the overall teachSng^^ff ectlveness ranking based 

on the assessment battery (COMPASS) correlated .49 with ah overall effective* . 

"it' ' 

ness rating based^on repeated classroom observations; but not with, an 
effectiveness rating based on an Intet^lew and not with any of the six pupil 
achievement-^test gain scores. The classroom observers' ratings did not 
cotrelate with the pupil gain^scoresy either; but the Intervlev/er's two ratings 
did correlate with the pupll-^gain scores in five out of 16 comparisons (.'41-^.58). 



A detailed .ccJmparisoA of the aVs^e^ssment ratings with the MAT gain scores 
revealed tKat the lack of co'rrelatiort was'dufe largely to seven of the ,27 cases, 
three of which were rated very high but had very low t)upil,^ain scores; the 
others were rated very low but had high .M^'iT gain scores, tatking out these 
cases, the other 20 cases correlated about .60; Preliminary scru^i^iy of 
these of f-'quadrant cases nsu^gests that the dimensions of *ef f ectiveneds oii^ which 
the asse$sors' and the classroom observets agree, but which are not related x 
to the MAT gain measures, may involve quite different kinds of teacher impact 
on pupils than the MAT measures; dimensions that are positively related to 
positive, confident^ initiative- inducing qualities in the teachers, as opposed 
'to the somewhat negative personal attitudes that were correlated with MAT 
gains (see Table 2). . • 

Indeed, in the case of the ten teachers with Title I classes, the assessment 
JtAges* ratings on such things as lack of def ensiveness, alertnesls. to classroom 
events, professional commitment, and attitude toward supervision were negatively 
correlated' with MAT gains' on ArithmetijC«Reasoning at an extremely high level, 
ranging from -.84 to •.9'5. Furthermore, this group showed ve'ry large negative 
correlati'ons between hIt gains on Arithmetic Reasotiing and such Biographical 

.f , ^ ..... ^ 

items as number of children .80) , father* s and mother^s education level 
(r,?5, -.95) -number 6f teachers in family (-.92), self-r'ating of health (-.80) 
and years of teaching (-.96^. Only number of graduate courses .correlated posi- 
tively with MAT gain (.77): ' ' ' ' 

Table 2 shows the results of -separately correlating ^thc; total set of ^ 
''predictc^r" measures objective scores on the Various assessment instruments, 
intirvJLewer ratings and observer ratings, with the pApil 'achievement- gain 
scores associated with each teacher, on ea<;h*of the fivei subscales of the 



Metropolitan* Achievement Test, in each of four years* Included in the table 
are those variables that showed three or more significant correlations ;^ut 

♦ ^ » 

of its twenty comparisons with. MAT scores* The assessment-battery score 
tbat»most often proved sighificantly related. to the >1AT gain-scores was 
the total score on the VieVs of >Xif e (negatively correlated). This indicates 
that the teachers whose pupils showed greater MAT gains tend to be psyctio- 
logically passive, cautious and lacking in self-confidence. The negative 
correlation 6£ their Emotional Control score indicates, however, that they 
say they prefer to accept and express their feelings rather than hold them in. 



The Adjective Self-Description scores show' ^at teachers with high pupil 

'f 

gains on the MAT describe themselves as unattracjbive in appearanc.e, and practical 
rather than idealistic* The Self-Report Inventory correlations show them 
expressing dissatisfaction with thanselves and'ali^o dissatisf^ption about the 
relationships they had W.ith their parents. „ . ^ 

'Factual items from the Biographical Form show that the more children of 
their own they have,- the less MAT-gains their pupils show; whe^jreas their pupils 
gain more, the more grad work the teacher has taken, and more she has ' 
earned a liasters degree. • * . ^ 

> The professional level-of-maturity saore on the Concerns instrument 
correlated second* md^st often -with the MAT gain-sjores. The more'^the overt 

topics they cited dealt with. concerns about pupil learning, *rather than self- 

A ' ' . . , 

prctpccupied concerns,, the better, their pupils did on the MAT gain criterion. 
However, when these same responses were evaluated "clinically," presumab'iy 
for their real*, underlying maturity, clarity and percfeptivefiess, both the* rating 
and the teacher- to- teacher ranking correlated negatively with the MAT scores, 

raising some question about the ger^cineness of the o^rtly expressed concerns, 

^ ' ' ( 

even though overt "professionalism" correlated with MAT gains. 



* As was true in Table 1, the interviewer's' ratings correlated « significantly 
. .and positively with MAI gains, in eleven out of forty comparisons* The class* 
^. room observer ratings also correlated positively with some MAT subscale gains, 
year by year, although they had not correlated significantly with the four- 
year , averages, in T^ible 1. . ^ ' . 

All of the "predictor" measures were taken in the f if fh year, after ttie . . 
four. years from which the pupil-gain scores were derived; so they were actually 
"post- die tors," Moreover, in the fouijjth year, four of the te^chers^^had changed 
from Title I to non-Title I 'schools a marked change in pupil population, 

while one teacher had switched the other way. The fourth year also saw a 

♦ » > ^ 

number of the teachers Change from the self-contained classroom .they had 
always known to a team- teaching arrangement. 

partly to test the possible effects, of wempoxal changes on the correla- 
tlons and partly to see if fourth«*year alterations particularly affected them. 
Table 3' was constructed. It shows that the significant correlations among the*, 
"pri^ictor** variables and th^ MAX g^n scores are hot randomly distributed 
across the four years. The closer- in time the MAI scores are taken, -within 
the first three years, *to the fifth-*ye^r observations and assessments, the 
larger is the number of significa^nt correlations, from 21 to 25 to 33« . 
the fourth year, :when numerous changes in type of teaching assignment) took 
place, the numbe^ of significant correlations drops to half that (17) of 
the third year. ^ . ^ ..." 

discussion ** ^ ' • • • 

• • • 

Two important findings *stand out. First, vdiateyjer airhievement test grains 

.^epr^ent as a desirable sign of pupil learning, and therefore of effective . 

^. ! ' ' • , . * 

teadhlng, they do not mefsure^ what ever it is that the classroom observers 

* ' ' ' 1 ^ 

' and the psychological assessor's ifiutuftlly agreed upon In this study^-^as im- ' •/ 
pottant aspects of effective teahhing (and* pup ir learning, by infer^nceX** • 



Second) those variables in the assessment battery whose s^cores correlated 
significantly with th&MAT gain scov^s fomed a highly consistent, and not ' 
al together. reassuring -parttern: self-doubting, psychologically passiye, 
^ somewhat unhappy women appe'^red more likely to generate high pupil gains on 
achievement tests,. Women with children of their own, confident of their own 
attractiveness and prone to cope with problems in an active, self*teli^nt 

• • • 

way did*aot produce large MAT gains. ^ 
To b'e'sure, the high^^ain teachers tended to have bore post<;baccalaureate 
education. They talked in a pupil-centered way on the Concerns instrument, 
although the .judge* who mlade a clinical evaluation of these data believed that 
thicse teachers quite often were voicing a "party line" of cliches, not 
'necessarily thinking hard' and' potently ab' it real, .specif ic pupil needs. - 

9 * 

Nonetheless, the data- did show a great deal of conscientious ♦intent and sincere 
efforFqn the part of ainjost every one of che teachers in the sample. ^ 

•• ' >^ ■ * ■ ■■ y ' ■ ' 1 ' ' '■■ ' 

* . Still, ,the evidence bo far in this st^ndy raises some^real questions 
about unctiticfally taking achievement test gains ^as 'a wholly desirable 
. educational yardstick.' Putting together the less than totally happy charac* 
teristics of the tea'thejcs who did induce such gains, with the fact that l>oth 
the c]rassroom,obsep^ers and the persohality assessors found some common agree** 

. . , • , • . ... 

ment on a presently unidentified kind of teaching effectiveness that is not 



at ail related to achievement test gains, tha data suggest that other criteria* 
of good teaching and good- learning ne^d to be identified and used. Beyond 
aoi^ie moderate point, it almost looks as though th9se who get children to 
learn, the somewhat mech4n.ical, atomized knowledge and skills tapped by * 
stand'ardized adhi'evement tests might unwittingly-^ deter other kinds of learning, 
creating a subtly depressing, low risk- taking, atmosphere that could conceivably 



keep cbiIc(^en*from learning, to cope. vigorously, sel£*relianc&y and happily 

*' • . ' • 

%ilth problems of It^aming^and -living. It cannot be too strongly remarked t 
tbls is much more /speculation tl\an repotting, at this point, but ve lat^^nd 

; ■ . ^ , . . ' • • - ; . . 

to pursue these leads, ^especially to identify the behavior that character4.2 
,^hose teachers whom the observers and assessors tholighc highly i'^fectlve, 

albeit that 'effectiveness was not reflected in achlevenent*test galas when 

* • ' ' • 

*that was the sole -criterion. * ' . * ' ' . /• * 

• > - ■ ^ •* 

• . • - • • • • ' V • 

'A methodological footnote of poss^ible interest derives from the ''ot^erv 
thiUu the closet in tiiae the criterion»and ^'predictor'' measures I were, (he mo 
closely, they correlated. This is scarcely a Startling new phenomenon; 
but It does imply that important changes In b^avlor and attitude do bccur, 

year by year, perhaps suf f itlently large to explain why siith a .small fractli 

* *. • * 

of teachers. (31 out of llSjisin this study) show 4 reasonably consistent pati 

... 

of pupil garins on the HAI, over only a three year span. 

^ . •.' ■ • . -• . •'^ 

Having hoted som^ possible dangers In using achievement test gains as t 
only critevlon, or perhaps even the doc^nant criterion, for Judging teaching 

' . A ' ' • . ' ' • • ^ • ♦ 

ejEf ectiveness,* the only re&noAslble stance is to make constructive suggesti< 
for a bettet mix of criteria. This will almost certainly continue to Incluc 
standardized testing of knowledge acquisijtlon« Uhat may be^ nee^^ are spfci 
procedures for helping teachers to work toward that kind of criterion Ih k 
non<-trlvial, non-mechanical, nonru;^a^ening way. Certain needei next steps 
In research seem to be rather <^learly yis^b^le in the present data. 
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Table 2 



Relative General Validity > Using the 
AchleveMnt**6aln Score Criteria 

20, separate teats: 4 years X 5 subscales of HAT 
Listed below arc^^e most successful predictors. 



Variabre 
VOL total score 
Cqncems level score 
9u8iber of children . 
Interviewer- effect iveness 
Interviewer-warmth 
ASD Attractiveness 
ASD Idealism 
Concerns Rating 
Concerns Ranking 
Mui&ber g^raduate courses 
Hastets degree 
-Observer- effectiveness 
SRI Self 
^SRI' Parent^ 
Vol .^motional control 



Significant 
8 

< 

7 
6 

.. 5 

;5 

• 4 
4 

4 . ^ 

4 

4 

3 , 

3 

3 



DiTection 

Negative C'P^usive") 

Positive 

Negative 

Positive > 

Positive 

Negative 

Negative (practical) 
-Negative 
Negative 
Positive 

I. 

Positive 
Pdsitive ' 
Negative 
Negative 

Negative (es^ressive) 



* Table 3 " 

Significant Correlations with All ^Tredlctors" 





Yl- 


t2 


Y3 


Y4 






2 
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.5 
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13 


WD 


' 2 


if 
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12, 
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3 
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15 


AC 
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3 
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1 


12 


AR 


11 


12 


13 
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44 




.21 


25 


33 - 


17 


96 



4 
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